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ABSTRACT 

An algorithm is proposed for obtaining local surface orientation from the apparent distortion of 
surface patterns in an image. 

A spherical projection is used for imaging. A mapping is defined from points on this image sphere 
to a locus of points on the gaussian sphere which corresponds to possible surface orientations. This 
mapping is based on the measurement of the local distortions of a repeated known texture pattern 
due to the imaging projection. This locus of possible surface orientations can be reduced to 3 unique 
orientation at each point on the image sphere using 3 vantage points and taking the intersection of 
the loci of possible orientations derived from each vantage. It is also possible to derive a unique surface 
orientation at each image point through the use of an iterative constrain', propagation technique along 
with the orientation information available at occluding boundaries. Both method are demonstrated for 
real images. 
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OVERVIEW OF THIS PAPER 

0.1 Main Result 
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Fig. 1 A picture of a golf ball. 

The distortion of these circles on the golf ball can be 
used to recover local surface orientations. 







Fig. 2 A obtained surface. 



/■"S 



Ik euchi 3 SFIAPE FROM REGULAR PATTERNS 



0.2 Summary of the Approach 1 

(l)We use a spherical projection to treat the perspective view. 
(A NEW TOOL TO DEVELOP A CONSTRAINT) 

We will propose to use a spherical surface as an image surface. This gives us an image surface perpen- 
dicular to all lines of sight from the viewer. 



(2) We propose a mapping which expresses regular-pattern gradients. 
(A STATIC CONSTRAINT TO BE USED IN THE PROPAGATION TECHNIQUE) 
Although Kcnder proposed two kinds of maps for texture gradients, neither of them can be applied 
to our shape-from-regular-pattern problem. The first one does not have the ability to express surface 
orientations from regular patterns on a surface. The second one requires aggregation of many lines to 
determine any one direction. Thus, we consider here the way in which the distortion of a regular pattern 
depends on surface orientations. 



(3) Our iterative propagation method for the shape-from-shading problem can also 
be used to solve the shape-from-regular-pattern problem. 

(HOW TO PROPAGATE CONSTRAINTS) 

We have proposed an iterative propagation method for the shape-from-shading problem. If we can 
describe regular-pattern gradients in die same way that the reflectance map expresses shading informa- 
tion, we can apply die algorithm to the shape-from-regular-pattern problem. Moreover, the shape-from- 
regular-pattcrn-problem always involves observable occluding boundaries required by the algorithm. 

(4) We also consider a technique for determining the surface orientation of a planer 
surface without iteration. 

(ENOUGH STATIC CONSTRAINTS CAN ALSO SOLVE A PROBLEM) 

If a surface contains more than three regular patterns and if we can assume that the surface is planar, we 

can determine its orientation without iteration using a method similar to the photo metric stereo method. 



Note that we will use two spheres. One sphere is the image surface, The other sphere (a Gaussian sphere) is used to express 
surface orientations. Both of the spheres are projected onto a plane using the stenographic projection for illustration in this paper. 
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1 A SPHERICAL PROJECTION 

We propose a spherical projection to treat the perspective view. An observer is located at the center of the 
sphere. All image contours are projected on this sphere with respect to its center. We will use the term image 
sphere as opposed to image plane in this paper to refer to the spherical projection. Every point in the real world 
is projected onto the image sphere with respect to its center. All lines of sight of the viewer are extended radii 
of the sphere and thus are perpendicular to the image plane. 




An Image 
sphere 



Fig. 3 An Image Sphere. 

All contours are projected on the image sphere with respect to the center. 
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All contours on die image sphere arc foreshortened by the relationship between the direction of the surface 
normal and die line of sight. At each point of the image sphere the line of sight is perpendicular to die image 
sphere, so there is no foreshortening of the image due to die position of a pattern on an image. 2 Thus, all 
contours on the image sphere arc due to the relationship between a line of sight and the surface normal of die 



The perspective projection is different from the spherical projection. In the perspective projection a pattern on an image is 
distorted twice. The first distortion is due to the relationship between the viewed surface normal and the direction of viewer's line 
of sight. The second one is due to the relationship between the line of sight and the image plane. Thus," the second distortion 
depends on the position of a pattern on the image plane. 
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surface patch viewed from the viewer. 

The essential point of a perspective view obtained with the spherical projection there exists only one viewing 
position, the viewer observes and interprets the outer world with respect to his position alone. He does not 
have to compensate for die orientation of an image plane. For example, consider the perspective projection 
of an extended planer surface parallel to die image plane which is covered with small squares (Fig. 4), 
The squares on die plane always project as squares, no matter where the squares lie on the surface. With 
die spherical projection the squares would appear as various rectangles depending on their positions on the 
surface. 



(a viewer) 



(an image plane of 
the perspective 
projection) 




(a planar surface covered by squares) 



Fig. 4 A Defect of the Perspective Projection. 

The so-called perspective projection cannot express foreshortened effect when an object exists on the plane 

parallel to the image plane. 



The spherical projection has several advantages over the conventional perspective projection. 

(1) The local geometry in an image docs not change when the orientation of the viewer changes. This is 
not die case with the conventional perspective projection where a change in the orientation of the image 
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plane can cause a dramatic change in the resulting image geometry. Thus, for the spherical projection 
distortion of a pattern depends only on the direction of line of sight and die direction of the surface 
normal. 

(2) The spherical projection can treat 360 degree scenes. 

(3) We can use the same regular-pattern gradient map at each image point in the spherical projection, 
because each contour depends only on the relationship between the direction of line of sight and the 
direction of the viewed surface normal. 

The spherical projection also has psychological validity. The most sensitive area of the retina (the forvea) has 
an extent of only 3-4 degrees over the visual field. Eve movements (and head movements) are required to 
examine patterns in different directions. Thus, the line of sight is always perpendicular to the image plane. It 
is interesting that some kinds of dragonfly have compound eyes which give rise to projection like that in the 
image sphere. Most photographs token by cameras cover no more dian a 20 degree field of view. Widiin this 
range, one can regard the image plane as a part of a sphere. 3 



Appendix 2 shows how to convert an image on the perspective projection into an image on the spherical projection. However, 
this adjustment, is usually unnecessary, because the extra distortion of the pcrspcclive projection is proportional to the cosine of 
the angle between the direction of the optical axis and the direction of the line of sight, namely cos 20 deg or 0.94. Thus, the 
distortion due to the perspective projection is small compare with the distortion due to the orientation of the viewed surface. 
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2 MEASURING TEXTURE DISTORTION DUE TO SURFACE ORIENTATION 
2.1 Definition of the Problem 

We will determine surface orientation based on the apparent distortion of regular patterns on an image sphere, 
provided that: 

(1) The surface is covered with a uniform texture of repeated texture element. 
We call the uniform texture as a regular pattern. 

(2) Each texture element is small compared with the distance between the viewer and the viewed surface. 

If the texture element is small, the texture element can be regarded as projected onto a tangential plane 
of the image sphere. All apparent distortion of the texture element is caused between the direction of the 
line of sight from the viewer and the direction of the surface normal. 

(3) Each texture element is small compared with a change of surface orientation there. 
Each texture element is assumed to lie in die plane of the viewed surface. 

(4) The original shape of the texture element (a generator) is known. 

Since we will decode surface orientations from the apparent distortion of a pattern, it is necessary for us 
to have a standard pattern with which to compare die distorted pattern. This requirement is similar to 
that of the albedo ratio in using the reflectance map. If one does not know the albedo ratio, one cannot 
make a correspondence between actual intensities and brightness values on the reflectance map. 

2.2 A Measure for Distortion in Regular Patterns 

In this section we examine the relationship between surface orientation and the distortion of regular patterns. 
Fig. 5 shows the projections of squares on a plane inclined 45 degrees in die vertical direction. Even though 
die generators (squares) lie on the same tilted plane, their images arc different from each other. Their distor- 
tion depend on two factors. One factor is the orientation of the surface. This is what we are interested in. 
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The other is the orientation of die squares in the plane of the surface. This in surface rotation causes the 
variations shown in Fig. 5. Our goal is to find some intrinsic measurement that depends only on surface 
orientation. Render's first method[l] treats only a few particular cases: when edges of the pattern are parallel 
to the coordinate axis and the edges are perpendicular to each other (only case A or case D in Fig. 5). Because 
the first mediod only takes a ratio of apparent lengths of two edges. The map cannot be applied to case B nor 
case C in Fig. 5, Render's second mcthodp] requires a large number of edge vectors in order to determine 
a single surface orientation. This method fails, when a surface has different orientations everywhere. For 
example, Render's second method will not decode surface orientation from the distortion of the small circles 
on the golf ball in Fig. 1. What we want is a way of measuring surface orientation independent of the in 
surface orientation of the texture elements. 
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Fig. 5 Projections of Squares ^ . ,. . . , 

All squares lie on a plane inclined in north-south 45 degrees and east-west degrees. Their distortion depend 
on both the orientation of the surface and the orientation of the squares m the plane of the surface. 
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If two perpendicular axis vectors can be associated with a texture element in a unique manner, we can use this 
pair of vectors as a kind of generalized texture element (See Fig. 6), When a pattern is distorted because of 
the angle between the direction of die surface normal and the direction of the viewer, the two vectors will also 
be distorted. This distortion of two axis vectors is die same as the distortion of the regular pattern, because die 
axis vectors are fixed on die regular pattern. We will, therefore, concentrate on the distortion of diese two axis 
vectors. 
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Fig. 6 

Pairs of axis vectors 
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The calculation in Appendix I shows us that the magnitude of the cross product of two axis vector projections 
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is proportional to cos a; and the sum of squares of their lengths is proportional to 1 + cos 2 a> where u) is 
the angle between the direction of the viewer and the direction of the surface orientation. These two values 
depend only on the surface orientation; and they are independent on the rotation angle of the regular pattern. 
For example, the patterns in Fig. 5 will yield the same value since they are all on the same tilted surface. 

The ratio of the cross product of the two axis vectors to the sum of the lengths of die two axis vectors is 
the desired measure for the regular-pattern gradient map. The length of a small object on the image sphere 
depends on the distance between the viewer and the object. A distant object is small while near objects are 
large. Both the cross product and the sum of lengths depend on the distance, but these effects cancel when 
their ratio is taken as shown in Appendix 1. Thus, the ratio is independent of the distance between the viewer 
and the pattern. This ratio is also independent of the viewer's orientation, because of the spherical projection. 
The ratio only depends on the angle between the direction of the viewer and the direction of the surface 
orientation. This means that we can use the same regular-pattern gradient map everywhere on the image 
sphere. 



We will call I in Eq. 1 the distortion value. 



P + 9 2 ' () 



where / and g are the observed lengths of axis vectors on the image sphere and r is the angle between the two 

projected axis vectors. /, g> and r can be measured directly from an image. 4 Thus, I can be obtained from 

a image, no matter how the pattern is rotated or reduced due to distance, / has the following relation with 

surface orientation; 

cose (2) 



1 + COS 2 LO } 

where uj is the angle between the direction of the viewer's line of sight and the direction of the surface normal. 

This/ will be the basis of the regular-pattern gradient map in the discussion to follow. 

/**\ If the two axis vectors have different three-dimensional lengths, this difference can be corrected for at the time of measurement 

r since the projected length of each vector is proportional to its actual length. Namely g' ~ /g where g is the measured length of 

the second axis vector and t is the ratio of the three-dimensional length of the first axis vector over the length of the second. 
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2.3 The Gaussian Sphere (A Method for Representing Surface Orientations) 

Surface orientations can be associated with points on the gaussian sphere. 5 The gaussian sphere is a sphere of 
unit radius whose Z-axis is taken as an extended line through the north and south poles of the sphere. Assume 
that we put a surface patch of an object at the center of the sphere and that the direction of the viewer is the 
direction from the center to the north pole. The surface patch faces some point on the sphere. For example, 
if a surface normal has the same direction as the south pole, the surface patch is perpendicular to the viewer's 
line of sight. All possible orientations of s surface patch are represented on the gaussian sphere. Thus, we can 
describe local surface orientation on an object in terms of points on the gaussian sphere. 

2.4 A Regular-Pattern-Gradient Map 



r^ 



We can associate the distortion value 7 with points on the gaussian sphere. Since the distortion value 7 
depends only on the angle w between the direction of the surf ce normal and the direction of viewer's line of 
sight. 7 is a function of the zenith angle of the gaussian sphere. Thus, we have circles of constant distortion 
values on the gaussian sphere. 

Fig. 7 shows our regular-pattern gradient map. 6 The curves shown are contours of constant distortion value. 
This map always applies to the shape-from-rcgular pattern problem. This is analogous to a situation in the 
shape-from-shading problem where the surface is known to be a Lambertian reflector and the light source is 
known to be near the viewer. 

2.5 Viewer-Centered Coordinate System and Spatial Coordinate System 

There arc two kinds of spheres: the image sphere and the gaussian sphere. The image sphere is a spatial 

coordinate system with which the image is specified. The gaussian sphere can be thought of as a kind of ruler 

for measuring s urface orientation relative to the north-south axis of the sphere. When a pattern is at the origin 
ITi is gaussian sphere is different from the image sphere. 

We project the gaussian sphere onto a plane using the stenographic projection. The stenographic projection puts the sphere on 
a plane and projects points on the sphere to the plane with respect to the north pole. Precise explanations arc given in [4,5,10]. 
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Fig. 7 The Regular-Pattern Gradient Map ■■ :! 

Each circle corresponds to a constant distortion value I. The gaussian sphere is projected onto a plane by the 
stereographic projection. 

of the image sphere, the two spheres are aligned. In other words, the viewer coordinate system on the gaussian 
sphere is same as the spatial coordinate system of the image sphere. When a pattern exists at some other point 
on the image sphere, then one must rotate the gaussian sphere so that its north pole points in diat direction (V 
in Fig. 8). 

Fig. 8 shows the relationship between the viewer coordinate system and the spatial coordinate system. 
(X, Y } Z) is the spatial coordinate system, (x, y } V) is the viewer coordinate system. The direction V denotes 
the direction of line of sight of the viewer, or the position of a regular pattern. 
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direction) 




* (a viewer coordinate system) 
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Fig. 8 Viewer-Centered Coordinate system (A gaussian sphere) and Spatial Coordinate System (An image 
sphere) 
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Formulas from spherical trigonometry give the following relationships; 

cos n = cos V cos u) -f- sin 6 V sin uj cos(/x — n — <f> v ), 



. , , , , sin w sinfu — tt — eL) 
sin(</>„ — 0„) = £— Z2z 



sin0 n 



(3) 



sinfl n sin5t, 

where w and p are the zenith and azimuth angles of the surface normal in the viewer coordinate system, 9 n and 
<t> n are the zenith and azimuth angles of the surface normal in the spatial coordinate system, and V and tf> v are 
the zenith and azimuth angle of the view vector in the spatial coordinate system. Thus, using Eq. 3 we can 
convert a; and /i into 9 n and <f> n . 

2.6 Obtaining Surface Orientation from Multiple View 
(How to Use Regular-Pattern Gradient Map) 

Surface orientation can be determined uniquely, given enough constraints. Surface orientation has two degrees 
of freedom. All points on the gaussian sphere are candidates before observation of a distortion value. The 
measurement of a distortion value, /, eliminates one degree of freedom, leaving a one dimensional set of 
possible surface orientations on the gaussian sphere. 7 More than two distortion values at die same point will 
uniquely determine the surface orientation there. This idea comes from the technique of the Photo-Metric 
Stcreo[3,4]. In practice, however, this texture stereo requires three different vantage points. 

There are two ways in which we can get more than three distorted values at a surface point. If there is one 
regular pattern on a surface, we can observe that regular pattern from several different vantage points. For 

It is possible in principle to determine surface orientation uniquely from the projections of the two axis vectors however we 
do not follow that approach in this paper favoring instead to explore the use of the technique of shading analysis for analyzing 
texture gradients. J " h 
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example, this case occurs when you pass a building, The pattern of the building changes according to your 
position.. We can also solve the problem from a single vantage point, if the surface is planer and contains more 
than three regular patterns, we can solve die problem from a single vantage point. Because the effect of several 
different vantage can be obtained by looking at different part of the planer surface. This case often occurs 
when we looks at a wall of a building that contains many windows. In either case surface orientation can be 
determined uniquely. 

The locus of possible surface orientation corresponding to a distortion value obtained at a point is a circle on a 
gaussian sphere. The center of the circle is the direction of die line of the sight in the spatial coordinate system. 
The radius of the circle is obtained from the distortion value, L Similarly, we will get another circle by using a 
different vantage point. The intersection of these circles is the solution. 



sible 
rface orientations 



radius is obtained 
from an observed distortion value 
and the regular-pattern gradient 
Vj ' map • 




e center point denotes 
A viewer direction. 



Fig, 9 Texture Stereo 

The center of the circle is die direction of line of sight in the image sphere (a spatial coordinates). The radius 

is obtained from the distortion value. 
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Since toe stereographic projection preserves circles, a circle on the image sphere projects to a circle on the 
stereographic plane. Thus, we can work with circles on the stereographic plane. Their centers are the projected 
viewing directions, and their radii can be calculated from distortion values, I. 

Fig. 10 shows photographs of a square piece of paper taken from three different vantage points. Distortion 
values hJi and h were calculated for these using Eq. 1. Namely, / = $gf , where f and g are lengths of die 
projected axis vectors on the pictures and r is the angle between two edges. FOr example, / = 25mm,g = 
37mm, and r - 131 degrees in the first photo, giving / = 0.35. Since we know the viewing direction, we 
can draw the circle 1. In the same way, we can draw circles 2 and 3. They intersect to each other at the point 
(0.0, 0.0) which gives us the surface orientation of the.square in the spatial coordinate system. 
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Fig. 10 A Real Example of Texture Stereo 

In case 1 f:25 g:37 uui:L31 azimuth :63 zenith:66 1:0.35, 

In case 2 f:47 g:49 tau:120 azimuth:45 zenith:55 1:0.43, 

In case 3 ' f:75 g:52 tau:S6 azimuth:. zenith :45 1:0.47. 

The third circle docs not go through the intersection of the first two, 

but one can see that -the accuracy is rather high given that real data 

is being used. 
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3 CONSTRAINTS AND THE PROPAGATION OF CONSTRAINTS 
(TO GET SURFACE ORIENTATION FROM A SINGLE VIEW) 

Constraint propagation is an important technique for problems involving analog values. Originally, this 
propagation idea was proposed for problems in designing electrical circuits by Sussman[6]. If a node in a 
diagram has a value, that value can effect surrounding nodes, reducing the range of possible values neighbor- 
ing points can assume. In other words, the value propagates through a network, and becomes a new constraint 
on the surrounding nodes. This propagation technique is more effective than the search technique, when the 
solution space has an large number of states. 

Smoothness gives us a tool for propagating constraints. If the surface is smooth, neighboring points must have 

nearly the same orientation. The smoothness constraint can be made explicit as a relationship between one 

j^\ node and its surrounding nodes. > n- 

3.1 The Smoothness Constraints (Propagation Constraints) 

Surface smoothness requires that the surface orientation be continuous over the image sphere. Our smooth- 
ness requirement is equivalent to the requirement that die surface height function be class c-1 on the image 
sphere [5]: 

1) surface is continuous on an image sphere, (c-0 w. r, t height) 

2) surface orientation is continuous on an image sphere, (c-1 w.r.t height) 

Eq. (5) is an "average" constraint which makes the smoothness requirement explicit. The definition of 
continuity can be expressed in a more convenient form; 



n 



A function F{ is continuous at (x 0) y ) 

if given an e > 0, 

there exits a 8 > such that when (x — x ) 2 -f (y — y Q ) 2 < S, 



\F t (x } y)-Fi(x 0) y )\<e. (4) 
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f-S 



If we take our mesh size S smaller that die 8, Eq. (4) guarantees that 
/orVe, 3S 

\H*o + $o> Vo) + Fi{xo - S 0> y ) + F^ y + S ) + Fi{x a) y - S ) - 4Fi{x 0} y Q )\ < 4e. (5) 

Thus, die difference of the surrounding values and four times the central value approaches zero as the mesh is 
made smaller. We call these constraints "average" constraints. A more precise discussion will be found in [5J. 

3.2 Regular-Pattern Gradient Map(Static Constraints) 
and A Propagation Algorithm 

The smoothness constraint can be used to reduce die locus of possible orientations on die gaussian sphere 
when only a single distortion value is obtained at each point. Each distortion value corresponds to a circle on 
f~\ the regular-pattern gradient map. Points on die circle are possible surface orientations at the point. We can 

make a propagation algorithm from die smoothness constraints to further reduce this locus of possible surface 
orientations. 
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Let 



where 



dR 



nr - ni n + <£)(/« - m }j n , cr u n ))(^\Fi, -.<*, -), 



dR 






77»* n 



FV-, , . 4- F 1 ? , . 4- F n 4- F n 



(6) 
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G i,3 ~ 
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F and G denote surface orientations. I itj is a distortion value at the point (ij) obtained from observation. R 
is the regular-pattern gradient map. Eq. (6) has three kinds of input; I itj (distortion value) comes from local 
observation, and the other two ( F* and G* )are propagated from surrounding nodes. In other words, this 
propagation algorithm makes a new estimate of the center node based on prior estimates of surrounding nodes 
and the observed distortion value( See Fig. 11) A more precise discussion on Eq. (6) is found in [5]. 
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Fig. 11 A Propagation Algorithm 

This algorithm is used iteratively. After the state of the estimation readies equilibrium, the algorithm terminates. 
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3.3 Occluding Boundary Constraints (External Constraints) 

At an occluding boundary we can determine the surface normal uniqucly[5]. The line of sight is tangent to 
the object at the occluding boundary. So the surface normal is perpendicular to the line of sight Since the 
line of sight of the viewer also lies at the tangent plane, die tangent plane is projected as a tangent line of the 
contour on the image plane. From the orientation of the occluding boundary we can determine surface normal 
uniquely at the occluding boundary 8 

This occluding information gives us die strong boundary conditions We need to start the propagation algo- 
rithm. The propagation constraints only convey information. In other words, the propagation constraints 
determine a relative relationship between nodes; but they cannot create new information. The occluding 
boundaries fix the surface orientation at those points. From these boundaries we can propagate the informa- 
tion inward over the image points adjusting the propagated value with the local constraints provided by the 
distortion value at each point. 

This propagation method is more suitable than Horn's method[8] in the shape-from-regular-pattern problem. 
While it is possible to use Horn's method which starts at a singular point, and proceeds along characteristic 
lines, his method is not able to use the occluding information, which is available in this problem. Thus, errors 
accumulate to much greater extent and the quality of the solution depends rapidly over long stretches. On 
die other hand, the method presented here utilizes constraints at both end point. 9 The use of the occluding 
boundary constraint makes our method is less sensitive to noise than Horn's. In addition the shape-from- 
regular-pattern problem provides fewer local constraints than the shape-from-shading problem. This shortage 
increases the need to utilize all available constraints. 



Note that [5] used the orthographic projection, while we use the spherical projection here. These projections are different from 
each other. However, the sphere projection has a image plane which is perpendicular to the lines of sight of die viewer everywhere. 
Thus, the same discussion is applicable to the occluding boundary. 

j**\ Actually, there are no start points and end points in our method. All information from occluding boundaries and a singular 

point is provided to the algorithm. Theoretically, however, a piece of information at an occluding boundary propagates inwards 
and interacts with information from the other occluding boundaries. 
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4 EXPERIMENTS 

4.1 Experiment 1 (A Infinite Planar Surface) 

Consider a plane covered by many squares. The squares are projected onto an image sphere using the spheri- 
cal projection. Fig. 12 shows the synthesized image. For graphical clarity, the image sphere is mapped onto 
a planar surface using the stereographic projection 10 This maps an infinite plane to the interior of the unit 
circle 11 . Note that the infinite horizon line maps to a circle. This circle is equivalent to an occluding boundary. 
Thus, the surface orientation is fixed on that circle. 
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lie squares are p« 
f an image sphere* 



Note that there are two spheres; the image sphere and the gaussian sphere, Both of them are projected onto a plane using 
the stereographic projection. 

'Hi is figure resembles a picture taken using a fish-eye lens. 
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The process contains three main stages (Fig. 13). The first stage obtains distortion values from a picture and 
surface orientations on the occluding boundaries. The second stage is die main one. In this stage we compute 
surface orientation for all points using die propagation method. Since the surface orientations obtained from 
the second stage are expressed in a viewer-centered coordinate system, a third stage is required to convert the 
result to die spatial coordinate system. 



an image 
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distorted values 
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the propagation algorithm 




re-adjustment 



\_ 



surface normals 
on the local 
coordinate system 



surface normals 
on the spatial 
coordinate system 



Fig. 13 Three stages 

(1) To obtain distortion values and surface orientations on the occluding boundary 

(2) lo compute surface orientation for all points. 

(3) To convert a viewer coordinate system to the spatial coordinate system. 
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We get the distortion using 



/ = 



fg sin r 

FT?-' 



where / and g are lengths of the projected squares edges and r is the angle between the two edges. Sec 
3.4 discusses the way in which surface orientations at the occluding boundaries is obtained. These distortion 
values and surface orientations are input to the propagation algorithm. Fig, 14 shows the distortion values and 
Fig. 15 shows surface orientations from occluding boundaries. 
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Fig. 14 The obtained distortion values 



Fig. 15 Surface orientations from occluding boundaries. 



The propagation algorithm estimates surface orientation at each point and is applied itcrativcly. at each node 
referring to the values of surrounding node and the corresponding distortion value. These estimates are the 
input information to the next iteration. When the entire system of nodes reaches a stable state, the algorithm 
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stops. 

There are two parts to the algorithm, a propagation part, and a regular-pattern gradient part. The propagation 
part conveys information from surrounding nodes to the center node. The regular-pattern gradient part uses 
the gradient map and a distortion value to determine the locus of allowed orientation. The mathematical 
details (convergence, effect of open boundary etc.) will be found in [5]. 

Fig. 16 shows the needle diagrams obtained for a planar surface. They arc expressed using the viewer coor- 
dinate system. The direction of the viewer's line of sight is taken as the z-axis, although the physical direction 
is different at each image point, because we are using the sphere projection. Fig. 17 is the planar surface 
depicted in the viewer coordinate system. 
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Fig. 16 The result needle diagram 



Fig. 17 The planar surface depicted 
in the viewer coordinate system. 
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Fig. 18 shows the same surface in the spatial coordinate system. The main source of error comes from the 
digitization of the third process. 
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Fig. 18.A View angle = tan"~ l 1.0 




Fig. 18.B View angle = tan" 1 10.0 



Fig. 18 The planar surface in the spatial coordinate system 
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4.2 Experiment 2 (A Golf Ball) 

We will compute the surface orientation for a golf ball using its surface texture. A golf ball has many small 
circles on its surface. The distortion of these circles can be used to recover local surface orientations. Fig. 19 
shows input the picture. 
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Fig. 19 A golf bail 

The distortion of these circles on the golf bail can be used to recover local surface orientations. 
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We can get distortion values from this picture by drawing two parallel lines in an arbitrary orientation tangent 
to each circle. We connect the resulting tangent points on each circle and use this connecting line as one of the 
axis vectors. We then draw a line parallel to the tangential lines through the center of the first axis vector. The 
part of this line within the projected circle is the second axis vectors. We can compute the distortion values 
using Eq. 1. Fig. 21 shows the distortion values obtained. 




r^ 



Fig. 20 How to get a pair of axis vectors from a circle . 




Fig. 2.1 Obtained distortion values. 

g* needle corresponds to a little circle on the ball nnd expresses how mud, the little circle is distorted 
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We can calculate surface orientation by using the algorithm. In this case the object is very small compared with 
the distance between the viewer and the object. Thus, there is of no need to convert the result in the viewer 
coordinate system to the spatial coordinate system. Fig. 22 shows the resulting surface. 
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Fig. 22 A computed results 
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5 CONCLUDING REMARKS 

(1) We proposed a sphere projection to treat the perspective view. 

(2) We defined a distortion value for a regular pattern and proposed a regular-pattern gradient map using 
this measure. 

(3) We showed that the propagation algorithm can successfully determine surface orientations from regular- 
pattern gradients using the sphere projection, the regular-pattern gradient map, and occluding boundary 
constraints. 

The propagation of constraints is a powerful idea in vision. Each node always has some relationship with 
the neighbors. Clarification of this relationship often gives us a key to the solution of vision problems. The 
propagation of constraints from neighboring nodes cuts down die area of possible solutions at the node. In 
vision the number of nodes is very large and each node takes analog values. This means that the solution ?nace 
has a large number of states. Thus, the propagation technique is more effective than the search technique. 
Even if the propagation technique cannot determine a unique solution, it can reduce the number of possible 
solutions so that a subsequent search technique can be used more effectively. 

There are several examples in vision where this technique has been used successfully. Marr and Poggio's first 
stereo algorithm used this idea [9]. Their continuity rule is an example of a positive propagation requiring that 
neighboring points have similar values (disparity values). Their uniqueness rule is an example of a negative 
propagation; A value at a node inhibits for another nodes from taking the same value. These two Riles 
describe the relationship between nodes. Ikeuchi's shape from shading algorithm[5] has the same form as the 
shape from regular-patterns algorithm presented here. Third, Horn's lightness reconstruction algorithm [11] 
also used a propagation technique. The G operator calculated a lightness value at the center node based on 
both old lightness values at neighboring points and output from a laplacian operator at die center node. 

The propagation of constraints technique is suited for array processing. The relationship between nodes 
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is homogeneous over an array. We can connect each node with the surrounding nodes in the same way 
everywhere, without regarding the position of nodes. An essential operation of the above mentioned algorithm 
is to compute an average of neighboring points. Thus, parallel hardware for array addition can implement the 
above mentioned algorithms very easily. The result can be obtained very rapidly. 
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APPENDIX 



A.l Derivation of the Distortion Value, / 
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Fig. A.i The relationship between the viewer and a regular pattern 
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Fig. A.l shows the relationship between the viewer and a regular pattern; the viewer is at V = (0 ; 0, cos^) 
and the texture element is at R — (— cos <f> v sin VJ — sin <f> v sin V) 0) 11 . Thus, the viewer vector V is 

V = (cos <f> v sin V , sin <f> v sin 9 V) cos 9 V ). 

The surface normal of a plane on which the regular patter lies can he denoted as 

N = (cos ^>sin^ ; sin <f> sin 9, cos 0) 

in the spatial coordinate system. We can take two base vectors of this surface as 

X = (sin <f>, — cos <f>, 0), 

Y = (cos </> cos 5, sin (f> cos 0, — sin 9). 



(A.l) 



X is a unit vector which lies on the intersection between die image plane and the surface on which the regular 
pattern lies. Thus, a general form for two axis vectors of die regular pattern is AJ3 where; 

A = (Xcosa -f- i^sina), 
B = [—X sin a-\-Y cos a), 

where a denotes the orientation of the texture element in the surface. Variation of this a causes changes like 
those shown in Fig. 6. 

-* -» -♦ 

Projections a and b of axis vectors,/! and B onto the image plane are 

& = k(A- (A . V)V) } 

b = k(B~{B'V)V) } [ ' } 

where k is a scale factor, which depends on the distance between the regular patter and the viewer. 

We want to find an intrinsic value dependent on the surface normal but independent of die rotation angle a 

and the scaling factor k. With this intrinsic value we will be able to construct a regular-pattern gradient map. 
'Hie following calculation will be easier if the z-axis is the same as the direction of the viewer. 
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Consider the ratio of the cross product of the two projected axis vectors over the sum of the square of the 
lengths of the two projections. 

The cross product of the axis vector's projections onto the image plane is: 

|a X 6| = k 2 \(V X X){Y -V)-(Vx Y)(X • V)|, 

= Ic 2 \V • {X X Y% (A 4) 

= k 2 cos a;, 

where co denotes the angle between the direction of die line of sight of the viewer and the direction of the 
surface normal. Thus, the cross product of the projected axis vectors does not depend on the rotation angle 
a, and is proportional to the cosine of the angle between the direction of line of sight of the viewer and the 
direction of the surface normal. 

/■^s The sum of the square of the lengths of the projected vectors is: 

\af + \b\ 2 = k\2-\\X-V\* + \Y.V\% 

= k 2 (i -\- lcosd v cos0-\-sin0 v sin0cos(<f> — 4> v )) 2 ), (A. 5) 

= & 2 (l-f cos 2 w), 

where we used a cosine formula from spherical trigonometry. 

We can cancel the scaling factor by dividing Eq.(A.4) by Eq.(A.5) obtaining the distortion measure /: 

|5xS| cos a; 
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A. 2 Conversion from the Perspective Projection to the Sphere Projection 

We can take two base vectors on the surface of an object. Every vector on the surface can be expressed using 
these two vectors. Let one vector be e L as on the extended plane of die meridian circle and let die other vector 
h as perpendicular to die first vector e\ . 



The second vector e 2 is projected in the same way in both the perspective projection and the sphere projection. 
The first vector ey is projected as in Fig. A.2. 
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Fig. A.2 
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Psphere = |§l|sino;, 

|ei| sinu; 



perspective 



cos V 9 



(A.7) 



where P 3p here and P P er 3 pective are the projected lengths of ey onto die image plane under the sphere projec- 
tion and the perspective projection, respectively, to denotes the angle between the direction of Viewer's line of 
sight and the direction of the surface normal. V is the zenith angle of the image point based on the direction of 
the optical axis of die image plane. 

Thus, 
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* sphere ==: ■* perspective COS 6^. (/4,8) 

Eq. (A.8)cxpresses the relationship between the sphere projection and the perspective projection. 

Eq. (A.7) also explains why the perspective projection causes the problem discussed in Chapter 2, Imagine an 
infinite plane perpendicular to the optical axis of the perspective projection. The infinite plane contains many 
squares. Under the perspective projection these squares always projected as squares. Which is contrary to our 
experience. We actually see squares as rectangles whose shape depends on dieir positions. In this case w is 
always as same as (f + 9 V ). Thus, from Eq. (A.7) we get 

* perspective === |C.l|» 

Thus, the perspective projection does not exhibit foreshortening effects. 



